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In 1995, the Appalachian Rural Systemic Initiative (ARSI) was 
awarded a 5-year grant to improve performance in mathematics and science in a 
66-county area in central Appalachia through high-quality, standards-based 
teaching supported by aligned, coherent local and regional systems. The ARSI 
model has three levels. The first level entails local reform and leadership 
development, involving local schools, districts, and communities. The second 
level consists of ARSI services and supports, which involve learning support 
services provided to districts by resource collaboratives located at five 
universities in Appalachia. The third level is the ARSI administration and 
planning level. ARSI contributions have been mainly in two areas. The first 
has been in building local district capacity for ongoing improvements by 
developing local leaders and a shared vision, aligning resources, creating a 
reform infrastructure, examining district policies, and developing resource 
collaboratives. The second area involves the quality of classroom instruction. 
Since ARSI has been aimed primarily at the professional development of lead 
teachers and district leadership, it seems reasonable that its influence would 
trickle down slowly into many classrooms. However, visible examples of good 
teaching are emerging. Of 54 classroom lessons observed, 40 percent were rated 
at the beginning stages of effective instruction and 11 percent displayed 
exemplary instruction. Yearly themes in the chronology of the project* s 5 
years are: laying the groundwork, establishing key resources in the region and 
districts, the changing vision of the role of technology, emergence of program 
improvement reviews as key strategy, and development of district-wide 
leadership teams. Challenges and suggestions for the future include sustaining 
the progress already made, developing community understanding and support, 
working within state contexts, and strengthening curriculum. (TD) 
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THE STORY OF THE APPALACHIAN RURAL SYSTEMIC INITIATIVE 



Introduction 

This is the story of the Appalachian Rural Systemic Initiative (ARSI), a five-year. 
National Science Foundation (NSF) funded effort to improve science and mathematics 
education in some of the poorest rural counties in the country. It is a story told from the 
perspective of a group of researchers who are "outsiders" to Appalachia and who were 
given the responsibility of documenting the work of the initiative. 

Our research group, Inverness Research Associates, was contracted by ARSI to serve as 
the external evaluators to the project. Over the past five years, our job has been to help 
document the work of the project, the realities ARSI staff have been confronting in 
doing their work, and the accomplishments of the project. Our work has largely 
consisted of visiting a sample of districts at select times over the course of the project, 
and providing feedback to ARSI staff based on our observations and interviews. To 
that end, we have written several evaluation reports detailing our findings . 1 

This project portrait is not meant to be another evaluation report. Rather we are 
providing this summary so that readers might have an abbreviated story of ARSI's first 
five years. We want to try to capture and portray the Appalachian Rural Systemic 
Initiative in such a way that outsiders can understand the journey the project has taken, 
the nature of the challenges the project has faced along the way, and the progress the 
project has made in its efforts to assist districts in improving their math and science 
programs. The information used in this portrait is gathered from our five years of work 
with ARSI, as well as from additional interviews with key project leaders and a "lessons 
learned" conference held on rural systemic initiatives . 2 

In the first part of this story, we discuss the origins of the project and give a 
chronological (and simplified) accounting of the project's efforts and evolution. In 
addition, we share some of the more general lessons learned along the way. In the 
second half of the story, we summarize our impressions of the progress the project has 
made, and the challenges that remain. 

Before we tell the ARSI story, however, it is important to discuss the region in which the 
project works. 

1 Reports include: "The Evaluation of the Appalachian Rural Systemic Initiative (ARSI): An Interim 
Report," December 1997; "Annual Report on the Evaluation of the Appalachian Rural Systemic Initiative 
(ARSI), August 1998; and "The Appalachian Rural Systemic Initiative (ARSI): A Report from the Field," 
April 2000. 

2 In April 2000, Inverness Research Associates conducted a Rural Systemic Initiative "Lessons Learned" 
Conference, funded by NSF. 27 representatives from the then eleven funded RSIs attended the four day 
conference, discussing their work in rural regions and the strategies that are most effective in promoting 
math and science reform in those settings. 
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The Nature of Appalachia 



While the Appalachian Mountains officially run from New York state to Mississippi, the 
region that is commonly referred to as "Appalachia" exists in the hills and hollows of 
Tennessee, North Carolina, Kentucky, Virginia, West Virginia, and Ohio. Many of the 
people who live in Appalachia have done so for generations. There is a palpable sense 
of pride among Appalachian people, as well as a commitment to community and 
family. In Appalachia there is a strong sense of "place;" people tend to grow up in one 
place and to remain there. Communities are small and people know each other well. 
While there is a shared sense of region, the people who live in Appalachia also tend to 
identify strongly with their own local communities and to value highly their local 
autonomy. The result is that it is often difficult to mount cooperative enterprises even 
within a single county. In these communities, poverty, isolation, and a lack of resources 
too often combine to create cultures where people suffer from low expectations and 
fatalistic attitudes. 

The isolation and parochialism of rural life in Appalachia are further exacerbated by 
deeply entrenched class distinctions, where for generations, the "haves" have controlled 
the access of the "have nots" to limited resources, power, and the political process. In 
1996, the poverty rate in the neediest counties in Appalachia was almost two times that 
of the rest of the country (24.12% vs. 13.9%). 

The isolation of the people of Appalachia persists because of factors that are both 
geographical and social in origin. The unique geography of the mountains seems to 
play a key role in everything that happens in Appalachia. Towns are isolated from one 
another, and even within small towns, various sections are spread out through several 
valleys, separated by mountains and connected by poor roads. In the winter, even 
small amounts of snow can shut down school for weeks at a time and further isolate 
people in their own small localities. Thus, the concept of community is not necessarily 
associated with counties or towns, but rather with small sections defined by 
physiographic features (e.g. the local hollows or "hollers"). In addition, the region is 
culturally isolated in that many of the cultural and educational opportunities taken for 
granted in more suburban or rural areas, such as universities, zoos and museums, 
simply do not exist here. The social origins of isolation arise from a sense of pride in 
their own communities and a lack of interest in the nearby "cities". There are also 
differences of language and culture that tend to keep people within their own local 
settings. 

The majority of jobs available in this region have historically been either hard labor (e.g. 
mining) or seasonal labor (e.g. farming). In recent years, advances in technology have 
left many Appalachian miners and workers unemployed. "Jobs are scarce, good jobs 
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very scarce, and virtually all jobs are controlled by a few powerful employers." 3 School 
districts are now the #1 employer in many rural Appalachian communities, with the 
result that the politics of school employment become an important factor in determining 
the nature and quality of education in the region. 

The school systems in Appalachia still have vestiges of a history that involves heavy 
elements of patronage. In addition, there is a great deal of ambivalence towards 
education itself in the community; those who succeed in school and go on to attend 
colleges and universities find themselves in a difficult situation where they typically 
have to leave the community in order to better themselves through higher paying and 
professional employment. Even if job opportunities exist for them at home, those who 
leave are often so changed by their experiences outside of Appalachia that it is difficult 
for them to come back. 

In our region in Appalachia , nobody goes back home. They leave if they get an education 
or if they are trained to do anything besides teach. . . You can almost get no doctors to 
work in Appalachia... Wehavea big shortage of engineers, a big shortage of trained 
people... because they have not had the culture engrained they don' t have the desire to go 
back 4 — RSI Lessons Learned Conference Participant 

Throughout the years, there have been many federal interventions in Appalachia to try 
to improve the socio-economic conditions as well as the functioning of the local 
education systems. Most of these efforts have not had lasting effects, and increasingly, 
outside interventions have been met by negative attitudes on the part of those whom 
they were intended to assist. As one person experienced in working in the region 
noted: "The challenges facing Appalachia today are rooted in a century of development 
that has left mountain communities economically dependent and lacking in the social 
and public resources for self-determination and sustainable growth." 

However, there is also increasing recognition that education is the key to long-term 
growth. Now that many of the mines are closed, there is a new possibility and 
opportunity for these isolated communities to develop themselves economically. Thus, 
at least in some communities, there is a recognition that a skilled workforce is key to 
local economic growth, and that good education, particularly in math, science and 
technology, is key to creating that skilled local workforce. 



3 Duncan, Cynthia M. World's Apart: Why Poverty Persists in Rural America . New Haven: Yale 
University Press, p. 17. 

4 The quotations here are extracted from phone interviews and focus group discussions. Some are edited 
and reconstructed, but we have made every effort to preserve their intended meaning. 
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The Origin and Goals of ARSI 

In the fall of 1993, the National Science Foundation funded a conference that brought 
together educators, community members and business people from Kentucky, Ohio, 
West Virginia, Tennessee, North Carolina and Virginia. This conference also convened 
national level funders and policymakers to look at the barriers for teaching mathematics 
and science in high poverty rural areas. After this conference, NSF sent out a request 
for proposals for funding Rural Systemic Initiatives, and in 1994, the Appalachian Rural 
Systemic Initiative (ARSI) received what amounted to a planning grant. ARSI used 
those funds to survey the educational landscape through regional conferences, study 
groups, surveys, and interviews with a variety of key stakeholders across Appalachia. 

Among other things, ARSI staff found in their early research that schools often served 
as nuclei for rural communities, playing a central and positive role in the economic and 
social life of many of these communities. On the other hand, they found schools with 
little capacity to improve themselves, limited resources within the small districts, and 
few connections to outside sources of support (e.g. large universities, regional and 
national organizations, etc.). And, despite the positive connotations associated with the 
schools, they found a lack of family and community involvement in the academic life of 
schools. 

When ARSI staff applied the NSF criteria for the RSI program, they found that there 
were 66 counties eligible in six different states that bridged the Appalachian region. 
Drawing on the research carried out during the development grant period, ARSI then 
designed an overall strategy for assisting those counties. Following the guidelines of 
the NSF systemic change initiatives, ARSI placed a high priority on designing an 
approach that would ultimately develop the capacity of the systems they worked with - 
so that local education leaders could initiate and sustain their own local processes of 
improvement in their own schools. In 1995, ARSI was awarded a five-year grant. 

Basing itself in Lexington, Kentucky, ARSI began to work with districts in the heart of 
Appalachia and wrote out as its mission the following: (ARSI is working to) "accelerate 
improved performance in mathematics and science for all students through high- 
quality, standards-based teaching supported by aligned, coherent local and regional 
systems." 

When I started this thing, I really wanted to improve the math and science abilities of the 
students, to make them more marketable - to make it easier for them to get jobs. I saw a 
great need for trying to boost the performance of all students and all teachers in 
mathematics and science In turn, this would help raise the level of the economy and 
socio-economics of the communities if these workers could attract new businesses. So it 
really was a bottom-up kind of deal - from the schools out to the communities. 

— Wimberly Royster, ARSI Principal Investigator 



INVERNESS RESEARCH ASSOCIATES 



Page 4 



In the original model for the project, ARSI sought to involve districts in math and 
science reform through the incremental development of leadership and the slow 
building of understanding and commitment of local school and community leaders. 

The key steps in this process involved: 

• Designating one school in each ARSI district to serve as a "catalyst school." This 
school would be a starting point for local improvements, provide a setting for the 
initial work of the Teacher Partner, and ultimately be models of reform in science 
and mathematics for the rest of the district. 

• Selecting "Teacher Partners" in participating districts. Project staff envisioned the 
Teacher Partners as "brokers" who would be liaisons for the project, connecting 
other teachers in their schools and districts through technology to professional 
development and materials. They would also learn from and contribute to other 
ARSI Teacher Partners through an ongoing ARSI supported network. 

• Providing resources and support to Teacher Partners via "Regional Resource 
Collaboratives". Created by the project, the Resource Collaboratives were housed in 
either major research universities or institutions within the target area, most with 
well-developed technology capacity and access to science and math expertise. Each 
had a full-time ARSI-supported director. Initially, they worked primarily with the 
Teacher Partners in their regions, brokering professional development, and creating 
and supporting a network of Teacher Partners. 

• Helping the schools use the Internet and computer technology as a way of 
overcoming the isolation of the region. ARSI staff, along with the Collaborative 
Directors, focused their efforts on helping districts learn how to use technology to 
increase the math, science and technology resources available to both teachers and 
students. The original model included working with a technology coordinator in 
each district to enhance the districts' technology infrastructure. 

• Creating a "community engagement team", headed by a community engagement 
facilitator, that could help to communicate to local community and school leaders 
the need for and nature of improved math, science and technology programs. ARSI 
staff believed that for math and science to become a priority in the schools, it wasn't 
just the school culture that would have to be addressed; members of the community 
would have to both buy in and be actively involved in these improvement efforts. 

Thus, the original ARSI model had three levels connected by the thread of technology. 
The first level was of local reform, involving local schools and districts, the community 
and a community engagement team. The second level was one of ARSI services and 
supports, which involved "learning support services" provided to the districts by 
Resource Collaboratives, and community engagement services. The third level was the 
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ARSI administration and planning level, including the ARSI administration and 
operations, the internal and external evaluators, and advisory groups. 

The Chronology and Evolution of the ARSI Project 

From 1996 to 2000, ARSI worked directly in 72 different catalyst schools in 52 districts, 
within 47 of the 66 targeted counties. Over the five years the project, the original model 
evolved as ARSI staff learned more about the region in which they were working, the 
needs of those teaching and learning in the region, and the relative success of the 
different elements of the ARSI model. In addition, as the project developed, new 
leadership and staff were hired. They had the experience, expertise and range of 
perspectives needed to continue to evolve the overall ARSI strategy so that it became 
increasingly effective at helping the local districts. 

In this section, we describe the key events that happened each year and how the overall 
ARSI model and strategies evolved. 

Year One (1995-96): Laying the Groundwork 

The ARSI project began in late fall of 1995 and was led by Wimberly Royster, Principal 
Investigator, and Dr. James Strom, Project Director. Much of this first school year was 
spent identifying the initial set of catalyst schools. ARSI contacted the superintendents 
of 88 school districts in 66 counties, laying out a general set of criteria for participation. 
The districts had to be willing to identify a catalyst school and a Teacher Partner who 
would be released half-time from teaching duties for the ARSI project. (ARSI paid for 
this release time and the support for the Teacher Partner constituted the major direct 
contribution that ARSI provided each district.) 

During the spring of 1996, the first districts were chosen. Over the summer, additional 
districts were recruited to create the first cohort of 21 school districts. Teacher Partners 
were chosen in these districts, generally identified by a principal or superintendent; 
some of these Teacher Partners were primarily technology experts rather than teachers 
with expertise in mathematics and science. Due to being identified in the course of the 
school year, some of the Teacher Partners were not released half-time during that first 
spring, but were given time to attend ARSI meetings and professional development. 

The Teacher Partners worked on technology improvements and shared ARSI-identified 
math and science resources with their colleagues, primarily within their own schools. 

In addition, three Resource Collaboratives were set up and staffed, at the University of 
Kentucky at Lexington; at Marshall University in Huntington, West Virginia; and at 
Appalachian State University, in Boone, North Carolina. In this first year, these 
Resource Collaboratives worked on helping Teacher Partners initiate their work, and on 
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setting up the networking and Internet connections with the first cadre of districts. 

They also assisted these districts in developing and writing "technology plans". 

Year Two (1996-97): Key Resources in the Region and the Districts Are Established 

In 1996-97, ARSI began to work in earnest with its first cohort of 21 districts. Much of 
the ARSI staff's efforts in this year were focused on continuing to support Teacher 
Partners in these districts, and helping the districts establish the technological capacities 
and components needed to carry out the work of the project. 

ARSI staff learned in their work with these first districts that to be successful, they 
needed to be accepted and to earn the trust of the district leaders. And the way to gain 
trust was, they found, to strongly support the development of indigenous leadership, 
largely in the form of the Teacher Partner. They needed to identify local leaders, first at 
the teacher and local administrator level, and build their capacity, before slowly 
working outward from this core. During this second year, the Teacher Partners began 
to develop their own skills and engineer appropriate roles for themselves within their 
own schools and districts. [ARSI staff soon realized that the progress of the local work 
depended largely on the skill and energy of the Teacher Partner, and ARSI staff wisely 
became much more specific as to the criteria to be used in the selection of the Teacher 
Partner.] 

ARSI staff also knew that one powerful way of overcoming the isolation that many 
Teacher Partners experienced was to connect them in a network. Summer institutes, 
monthly meetings, and professional development experiences, as well as ongoing 
contact via e-mail and visits from Resource Collaboratives, helped create a solid first 
cadre of Teacher Partners. These individuals enjoyed and benefited from working with 
their colleagues who came from across Appalachia. 

In this second year, ARSI also continued to work on the notion of the catalyst schools 
that were meant to serve as models for improvement for the districts. The catalyst 
schools were (and still are) those schools where the Teacher Partner centers his or her 
work. The original intent in the ARSI model was that the catalyst school would be 
selected first, then the Teacher Partner would follow from that. As it played out in the 
first few years of the project, the Teacher Partner selection often came first, and the 
catalyst school was the school in which that teacher taught. 

Technology was a key focus in the first few years of the ARSI project. ARSI staff 
believed that through technology, they could help districts and schools overcome the 
geographic isolation of the region. ARSI staff envisioned that using technology, 
teachers would connect with one another and with a broader professional development 
community. Moreover, ARSI strongly intended that the Teacher Partners and Resource 
Collaborative Directors would help schools and districts use technology to find a wide 
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range of resources that could be used to improve local mathematics and science 
programs. 

Near the end of year two, a staffing change occurred at the administrative level, with 
Dr. James Strom leaving the director's position, and Dr. Keith Smith replacing him. 
Smith brought a focus on curriculum and district-level leadership to the project. His 
background in administration helped focus ARSI's attention on a new key player to be 
brought to and included in the ARSI model: the District Liaison. 

While most district Superintendents liked what the Teacher Partner was doing within 
their own schools, they did not essentially see the ARSI work as intended to promote 
district-wide systemic changes. Thus, it was important for ARSI to continue to promote 
the vision of district-wide reform and to get the commitment and involvement of at 
least one high-level district administrator. This District Liaison was meant to champion 
the work of ARSI district wide. This meant, more specifically, providing direct support 
for the work of the Teacher Partner, being a public advocate for ARSI's efforts across the 
district, and helping to identify and pull together additional resources to carry out 
improvements in mathematics and science. In addition, the District Liaison was 
typically positioned well to identify the district-level policy supports needed to sustain 
improvements the mathematics and science programs. The project director explained 
the importance of identifying this key player, "We knew that in every district you will 
find one very capable or very political, and sometimes both, person... And the key was 
to find and train this person. " 

Year Three (1997-98): The Role of Technology Changes 

In 1997-98, ARSI expanded its work to include 39 catalyst schools in 36 districts. Several 
shifts in focus and strategy occurred during this time. ARSI began to downplay the 
primary focus on technology. Teacher Partners from the first cohort began to branch 
out and work with teachers in other schools in their districts. The District Liaisons were 
given more attention and explicit training. Resource Collaboratives moved more 
toward directly providing services in addition to brokering existing services. Districts 
started to pool funding from different sources in the service of math and science 
improvements. Also, a process for reviewing schools mathematics and science 
programs was initiated in some ARSI districts in Kentucky. 

In this third year, the change in the vision of the role of technology was a key turning 
point in the history of ARSI. A year and a half into the project, two key things 
influenced this shift. The first was a finding from a NSF site visit, that NSF staff felt the 
ARSI project was focused too heavily on technology implementation and not enough on 
the reform of mathematics and science teaching. The second influence was that the six 
state departments of education had made available funds for districts to begin the 
technology improvements. With technological improvements coming from other 
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sources, the project decided to turn its focus more toward instructional issues in math 
and science education -- and less on the use of technology to gain access to resources. 
Thus, there was a shift from seeing itself as a program that used technology to locate 
resources, ARSI now looked more toward implementing standards-based mathematics 
and science instruction in the classrooms of Appalachia. This shift had both negative 
and positive implications. 

On the negative side, those involved in ARSI from year one felt that ARSI was sending 
them mixed messages about the goals of the project. Staffing changes at the project and 
Resource Collaborative level were made to adjust for the shift; in some cases, 
technology coordinators were replaced with staff people who had greater expertise in 
science and mathematics curriculum. On the positive side, focusing on math and 
science education led to greater clarity about the direction of future ARSI efforts that 
would be needed to bring about these broader reforms. The role of technology in ARSI 
now focused on learning how to integrate technology into the classroom in the service 
of improving math and science instruction. 

This is also the year in which ARSI was first faced with a decision about the long-term 
involvement of Teacher Partners. ARSI intended that once a school or district was 
involved in the project, they would stay involved, but ARSI would only pay for the 
release time for the Teacher Partners for two years, and (they hoped) the districts would 
pay for their release time after that. ARSI staff learned that after two years, most 
districts weren't invested enough in the project to make that kind of commitment. ARSI 
also knew that they had, by now, a very committed and strong core group of Teacher 
Partners that were important to the project. Consequently, ARSI decided to continue to 
fund the release time of the those Teacher Partners in the first cadre whose districts 
were not positioned to cover that cost beyond the initial two-year time frame. 

ARSI began to see variations in how districts were using Teacher Partners and how the 
Teacher Partners were positioning themselves within their own districts. Some Teacher 
Partners were released full-time, with ARSI picking up half. Other districts decided to 
fund two half-time Teacher Partners -- with each working at different schools. The 
commitment of ARSI to continue its support for the first year cadre of Teacher Partners , 
in effect, limited the number of additional catalyst schools that ARSI could target in the 
region. ARSI decided that there was more value in extending and building upon the 
work of their trained Teacher Partners than in trying to cover all the counties in their 
target area. They chose, in essence, depth over breadth and in doing so acknowledged 
the need for a long-term capacity building process in each district. 

Resource Collaboratives, brought on board at the end of the second year — University of 
Tennessee in Knoxville; Ohio University in Athens; and at Clinch Valley College (now 
the University of Virginia's College at Wise) -- became fully operational in the third 
year . In addition, Appalachian State University phased out as a Resource Collaborative 
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with the addition of these new collaboratives who were located in more direct 
proximity to ARSI districts. 

The role of the Resource Collaboratives also shifted. As the project evolved/the 
Resource Collaboratives became more active in directly providing services and 
technical support. Thus, rather than solely helping districts use technology to connect 
to greater resources, the Resource Collaboratives took on a more ambitious role, 
focusing on helping teachers use technology in the service of good instruction. 
Additionally, Resource Collaboratives also focused their work on national standards 
and inquiry-based instruction. In all of these areas the Resource Collaboratives focused 
on building and fostering the Teacher Partners and District Liaisons as a team in each of 
their districts, looking increasingly at the ways in which the district as a whole could 
improve their overall math and science programs. 

It became apparent that there were issues beyond the classroom - issues of time, 
resources, support, stability, and direction - when there was a real need for someone with 
administrative ties to be involved in the project in a more direct way. So over time, the 
Resource Collaborative moved from their initial teacher orientation, which had been very 
beneficial at the start, into a larger focus on building and sustaining these leadership 
teams in the districts. - Keith Smith, ARSI Project Director 

At this time in the project, the Resource Collaborative in Kentucky began to implement 
a strategy for reviewing science and math programs in their districts. The Resource 
Collaborative in Kentucky had brought in a team from the Central Kentucky 
Educational Cooperative (CKEC) to help their districts with curriculum work. This 
team. Dr. Steve Henderson and Dr. Ron Pelfry, developed original instruments, based 
in part on the national standards in science and mathematics, to conduct curriculum 
audits specific to math and science. From this work the suggestion was made that the 
other ARSI schools in Kentucky might be interested in and benefit from these audits. 

Year Four (1998-99): The Program Improvement Reviews Emerge as a Key Strategy 

In 1998-99, ARSI worked with 57 catalyst schools in 44 districts. Increasingly, Teacher 
Partners evolved from school-wide to district-wide roles. Some districts were 
successful with their efforts to engage the community in math and science improvement 
efforts, while others continued to struggle with this component. District Liaisons began 
to pay more deliberate attention to their use of other resources and funding for math 
and science, and to policy issues that affected their math and science programs. The 
focus of the project in the most advanced districts turned more toward the development 
of district-wide leadership teams. ARSI also looked closely at the curriculum audits 
began in Kentucky, and began to make that process available throughout ARSI. 
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Perhaps most significantly, as the Kentucky Resource Collaborative began the program 
improvement reviews, ARSI staff became aware that in most districts, the mathematics 
and science curriculum was, at best, put together in a piecemeal fashion. Curriculum 
was not aligned across the grade levels K-12, let alone aligned with national standards 
in science and mathematics. The notion of a district-wide science and mathematics 
program was largely absent. Therefore, ARSI needed a vehicle for helping districts take 
a broader look at the current state of instruction, curriculum and instructional materials; 
and the degree to which these matched a national vision for teaching science and 
mathematics. The curriculum audits seemed to be a good tool to accomplish this goal. 

Thus, a key strategy for ARSI emerged - one that has since become more widely used 
throughout the project. The ARSI staff began referring to the audits as "program 
improvement reviews," so the districts would not feel the project was "checking up on 
them." To date, the program improvement review process has been adapted to and 
used in Kentucky, Ohio, Tennessee, Virginia, and West Virginia. 

During these program improvement reviews, teams composed of ARSI leaders. 
Collaborative Directors, District Liaisons and Teacher Partners would visit and review 
the programs of other districts. These reviews are followed by the creation of action 
plans and follow-up site visits to help ensure that something productive happens. As 
the project director explained, these reviews proved to be beneficial in a number of 
ways: 

First, these reviews gave people data about their school that was put together by people 
outside of their school. The people who did the reviews were trained, and they often 
learned a lot from those visits, getting ideas from what others were doing and reflecting 
on their own programs back home The teams also made specific recommendations about 
the kinds of improvemmts that were needed, not just saying "You need to improve your 
math program " So as the prq'ect went along the program improvement reviews really 
became much more central to the work we were doing 

The development of leadership teams 

The concept of building the leadership capacity outward continued with the addition of 
superintendents and principals to the leadership teams in each district. These 
leadership teams attended summer academies together to focus on the work of 
improvements in science and math, and how to extend that work beyond the 
classrooms of the Teacher Partners -- and beyond the walls of the catalyst schools to 
other schools in the district and the community. Thus, the model which had started as 
one of ARSI and Resource Collaborative staff providing the leadership to Teacher 
Partners and District Liaisons evolved into ARSI and Resource Collaborative staff 
helping to pull together, and then provide support for, a district team of leaders. 
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Also, as the project continued to evolve, the growing group of ARSI Teacher Partners 
grew into their roles developing into leaders of both local and regional reform efforts in 
mathematics and science. In several districts, these Teacher Partners became the de 
facto, and in some cases, the official, district math and science supervisors. In their 
individual districts, some of these Teacher Partners built a supportive group of teacher 
leaders within their districts -- creating mini networks of teachers and local leadership 
teams who would meet regularly to discuss ideas. Many of the Teacher Partners as well 
as other teachers in their districts also were gaining access to and participating in 
extensive professional development within their states and region, and on a national 
level, (e.g., attending nationally recognized programs at the Exploratorium and the 
Woodrow Wilson Institute). 

Community engagement 

In this fourth year, the community engagement piece of the ARSI model continued to be 
one of the most challenging pieces of the model to implement. While some schools and 
districts embraced the idea of greater community involvement in math and science - 
and were able to reach out to the community through innovative events, such as science 
nights - the majority of Teacher Partners and community engagement facilitators found 
it difficult to engage already busy members of the community in an endeavor whose 
purpose was vague at best. Teacher Partners and Resource Collaboratives were so busy 
with their professional and leadership development and curriculum work that often 
there was little time for the community piece. Also, the amount of time ARSI had 
contracted for staff to lead the project's community engagement efforts, was insufficient 
for such a big task. ARSI ultimately did develop a guidebook to help districts conduct a 
needs assessment on the issue of community awareness of and involvement in math 
and science improvement efforts. 

The position of community engagement facilitator also becomes somewhat problematic. 
Originally, the community engagement facilitator was intended to be someone from the 
community. However, in several districts, the Teacher Partner or another teacher was 
asked to be the community engagement facilitator. Thus, what was to be a community- 
led endeavor became dominated by school personnel in a number of districts. 

Readiness for reform 



As more districts came on board, ARSI became increasingly aware of the variations 
across the districts in terms of their overall functionality and hence their "readiness" for 
reform. Several districts were fairly "reform savvy" from the start and had individuals 
with the knowledge and sufficient expertise to push for reforms. These districts were 
functional and strong enough to encourage these individuals and find resources to 
support their work. Other districts were just barely surviving, able only to focus on 
providing the basic needs of safe school buildings and transportation. There were no 
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individuals to take the lead, and not much encouragement from the district leadership. 
In these districts, the infrastructure and community support for schooling was 
negligible and there were virtually no capacities for reform or improvement in place. 

Throughout the course of the project, ARSI found appropriate ways to intervene and 
assist in every type of district. They modified their approaches along the way to keep 
their overall vision while matching their strategies to meet the needs and realities of 
individual districts. In the districts that were farther along, where there were 
leadership teams already in place, ARSI staff focused its efforts on helping them more 
clearly articulate a vision for improvements, or connecting them with national-level 
funding to help take their ideas to the next level. In the districts that were barely 
surviving, it meant finding good teachers and providing them with the supports and 
resources needed to become a school and district-wide advocate for reform. 

Year Five (1999-2000): District-wide Leadership Teams Develop and the Project Looks 
to the Future 

In 1999-2000, ARSI continued to work with 57 catalyst schools in 44 districts. Much of 
the work was similar to that done in year four; at the local level, efforts focused on 
conducting and following up on program improvement reviews and building 
leadership teams in the districts. Some districts were just beginning their efforts with 
ARSI, and others were benefiting from long-term participation. ARSI also focused more 
on gathering documentation of its first five years of progress, and on how to sustain 
their work thus far into the future. 

The project also experienced turnover at the director level. Keith Smith left the project 
and Steve Henderson, came on board as the project director. As before when leadership 
changes happened, the Principal Investigator of the project proved to be very sensitive 
to the needs of the overall project in selecting new leadership. For example, Keith Smith 
came on board when the project began to move from a technology to a district focus. 
Similarly, Henderson brought his expertise with the program reviews and leadership 
team development when the project began to focus more on the work of district 
leadership teams. These careful hires were one of the things that have made the project 
as successful as it has been. It was able to take advantage of the opportunity that 
leadership changes provided to find people with specific characteristics that could best 
meet the needs of the evolving project at the time they were hired. 

Project staff also focused more attention on documenting the overall impact of the 
project. Henderson focused on gathering both local and regional data not only to 
demonstrate the project's impact to NSF and other stakeholders, but also to reinforce to 
district personnel the importance of data-driven decision making. 
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The project also looked to the future, developing a proposal for a second round of 
funding from the National Science Foundation in order to continue their work in the 
region. 

The Contributions of ARSI 

We now turn our attention to the contributions ARSI has made to date. Over the past 
five years, our studies have involved annual intensive site visits to ARSI districts. Our 
most recent site visits to districts 5 focused on gathering data about how ARSI has 
contributed to local district and school capacity for sustaining the process of math, 
science and technology reform. We also carefully documented the quality of classroom 
practice in "ARSI" classrooms 6 within catalyst schools and districts. During these visits, 
Inverness researchers interviewed the key players in the ARSI effort within each 
district, and observed 54 math and science lessons at the elementary, middle and high 
school levels. 

In this chapter of the story, we will draw on the data from our final site visits and 
discuss the overall progress of ARSI and the status of ARSI districts, schools and 
classrooms in year five. We will discuss the project's influence on local district capacity 
for reform 7 , as well as the project's influence on classroom practice. In the final section, 
we will examine the challenges the project faces as it strives to continue its efforts in 
Appalachia. 

Building Local District Capacity for Ongoing Improvements of Math and Science 
Programs 

The ARSI project has clearly made a significant contribution to the internal capacity of 
participating districts. The six districts we visited in the fall of 1999 continue on an 
upward trajectory as they work to improve their math and science programs. ARSI is 
helping each district move forward, no matter their initial starting point. In addition, 
we found the districts to be well positioned to maintain and even continue to build their 
internal capacities in the coming years. Hence, we saw in the ARSI districts the 
beginnings of the long process of developing the leadership and the critical supports — 
such as good curriculum, well-designed materials, professional development and 



5 In the fall of 1999, we visited six "best case" districts, selected for the noticeable strides they had made in 
their math, science and technology reform efforts. Four of these were part of the first cohort of districts, 
and four of these districts were ones we had visited previously. 

6 These classrooms were defined as those where: 1) the teaching is beginning to reflect the national 
standards and the qualities of teaching and learning that ARSI is promoting; and 2) where the teacher has 
been "influenced" by ARSI (i.e., attended ARSI professional development offerings, spent time working 
one-on-one with the teacher partner, used the teacher partner as a resource in some way, etc.). 

7 For a more detailed and thorough discussion of the study of A RSI's contribution to district capacities 
and classrooms, see Inverness Research Associates' report "The Appalachian Rural Systemic Initiative 
(ARSI): A Report from the Field." 
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appropriate assessments -- that were needed to initiate and sustain solid, well-designed 
mathematics and science programs. 

In what follows we briefly describe the contributions of ARSI to key capacities that are 
requisite to further improvement efforts. 

• The development of indigenous leadership 

ARSI is a subtle reform effort that is steadily building within each district a grassroots 
group of teachers and district leaders - people who are knowledgeable about and 
increasingly advocates for inquiry-based, student-centered, hands-on teaching and 
learning. In most districts, a point person for science and mathematics reform would 
not exist without ARSI. The project has provided key supports to Teacher Partners so 
that they can grow in their skill and expertise. In addition, most of the districts we 
visited now have a core group of teachers and administrators who provide visible 
support and motivation for improving science and mathematics education. ARSI has 
also been a key vehicle in creating and supporting networks. Tire Teacher Partners and 
District Liaisons are now better connected in supporting each other's leadership. 

• The development of shared vision 

Through their work with ARSI districts and communities have become more committed 
to the idea of math and science reform. Through the dedicated work of the Teacher 
Partners, and increasing participation in ARSI by district administrators, most of the 
districts we visited had developed a more serious intention to improve their math and 
science programs. 

Just as important, ARSI has allowed leaders in ARSI districts to know the reality of their 
districts better. The work of ARSI has led to greater awareness of both strengths and 
weaknesses and thus led to clearer understanding of the need to improve math and 
science teaching. Moreover, ARSI has provided district leaders with a way of thinking 
about and discussing science and mathematics education that is more in line with 
national standards. ARSI has helped these districts know what they do not know, and 
develop a more sophisticated vision of good teaching and seeing where they are now in 
that light. 

• The alignment of resources 

In terms of ARSI's influence on the reform infrastructure in the districts, all of the 
districts we visited were resourceful in grant writing and obtaining the resources that 
are made available in many states to economically poor, rural schools. In addition, 

ARSI has encouraged and supported districts in converging resources, such as Title 1 
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and Title 2 funds, so that they use existing ones to support math and science reform in 
an increasingly coherent fashion. 

• The creation of a "reform infrastructure" 

Overall, ARSI has initiated the process of helping districts develop infrastructure. ARSI 
has had some influence on districts' knowledge about and exposure to "exemplary" 
curriculum; however, none of the districts we visited had officially adopted or 
implemented any of these curricula. We should note, also, that the program 
improvement reviews in Kentucky are leading districts to investigate more seriously 
changes in curriculum within schools and across districts. Also, in part because of 
ARSI's efforts and in part because of the increased focus and funding from the states, 
technology - equipment and software - is evident in all districts. 

• The examination of district policies 

Perhaps it is not surprising that we found ARSI's influence on broader policies that 
affect math and science to be quite limited in the districts. None of the districts we 
visited had instituted their own standards, assessment or policies vis-a-vis math and 
science education. Rather, they took the lead from the state they were located in. 
However, many of the districts we visited have policies in place that can be broadly 
supportive of reform, such as block scheduling, common planning time for teachers, 
and release time for professional development. And perhaps the most important 
district policies we noted were centered around the ongoing support for the Teacher 
Partner to continue working with ARSI, and the commitment to sustain a process of 
improving mathematics and science education within the district. In addition, those 
districts who are farther along in their reform effort were more likely to be proactive 
and thoughtful with regard to the impact (positive or negative) of the district policies on 
reform efforts. 

• The development of resource collaboratives 

Finally, the Resource Collaboratives have also been important for ARSI's success. 

While the role of the Resource Collaboratives has developed over time, ultimately, they 
have played a key role in assisting districts in the hiring and training of teacher partners 
and district liaisons and in providing them with on going professional development 
and support. In addition, the Resource Collaboratives have increasingly encouraged 
and guided ARSI districts to focus their work on national standards and inquiry-based 
instruction. 
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Influencing the Quality of Classroom Instruction 

The second dimension of ARSI's contribution we want to discuss focuses on the nature 
and quality of classroom instruction. In examining the effects of ARSI on classroom 
instruction, it is important to keep several things in mind. First, ARSI's primary focus is 
not on classrooms but on building the capacity of and the leadership within districts so 
that they can understand, initiate, and sustain reform themselves. Second, ARSI is a 
relatively "thin" initiative, investing few dollars per student and teacher when the effort 
is "amortized" across all the districts it is serving. Third, ARSI has only been a presence 
in Appalachian districts for a few years. For all these reasons, it seems reasonable that it 
may take some time before the influence of ARSI "trickles down" into many classrooms. 

However, our visit in the fall of 1999 indicated that the capacity building efforts of ARSI 
are beginning to influence classroom practice. In the classrooms we visited, the efforts 
of the ARSI project have meant that there are good teachers who are beginning to use 
ideas that are supported by and come from ARSI in their classrooms; simultaneously, 
there is increased recognition of and support for teachers who are already teaching in 
accordance with a standards-based vision. 

We used an instrument developed for the NSF LSC initiative to assess the quality of 
ARSI classrooms. Overall, of the 54 lessons we observed, the most common lesson we 
saw (40%) was rated a "3" - at the beginning stages of effective instruction. 8 These were 
classrooms that were beginning to use cooperative learning, beginning to have student- 
centered instruction, and perhaps beginning to use a curriculum more in line with 
standards-based practices. Eleven percent of the teaching we saw displayed 
"exemplary instruction"; these teachers were doing wonderful lessons and were well 
supported by ARSI. However, 43% of the classrooms we visited were still below the 
beginning stages of effective instruction, and 17% of these were very weak. 

Overall, in the classrooms we observed, lesson content came closer to reflecting best 
practice than either lesson design or implementation. In addition, the math lessons we 
observed were farther along in reflecting best practices than the science lessons. Almost 
half the lessons we observed were rated as having a positive impact on students' 
understanding of important science and math concepts. However, while the language 
of inquiry-based instruction is beginning to infiltrate the districts, the classroom practice 
is still more traditional in nature, even in those classrooms where ARSI feels it has had 
the greatest influence. 

Thus, ARSI, even though it has been aimed primarily at the professional development 
of lead teachers in these districts, is beginning to recognize and contribute to the nature 

8 The rating instrument is designed specifically to measure the extent to which classroom practice reflects 
the vision of math and science instruction laid out in the national science and mathematics standards 
documents. 
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and quality of teaching and learning in the "ARSI" classrooms. While good instruction 
does not perhaps run wide or deep in these districts, given that the ARSI districts are 
faced with challenging circumstances, there are visible examples of good teaching, and 
these teachers are being recognized and supported by ARSI. 

The Challenges That Remain and Suggestions for the Future 

ARSI has made significant strides in the first five years of its funding. However, the 
region is large, the challenges are difficult, and much remains to be addressed if the 
efforts of the project are to make long-lasting contributions to the region. The challenges 
for ARSI are multiple. There is the sheer size of the area it is working with and the large 
number of isolated districts and schools it is trying to serve. ARSI districts are all at 
varying stages of development and readiness; also, ARSI has three cohorts of schools 
and districts, each of which has variation in the amount of exposure to the project. 

There are the difficulties it has in working with six different states and six different state 
level departments of education. And there is the ongoing challenge of working with 
districts, schools and teachers who have a strong sense of place, a commitment to 
maintaining local values and traditions, and who are, therefore, justifiably skeptical 
about the assistance of outside agencies. 

However, we should also point out that ARSI now has assets in place that it did not 
have five years ago. There is a strong network of competent and committed Teacher 
Partners. There are administrators who both understand and are willing to promote the 
work of ARSI. There are Resource Collaboratives who have developed productive and 
ongoing working relationships with many of the poorest districts in Appalachia. And, 
finally, there are structures and ways of working -these have evolved over time and 
through experience, and they provide ARSI and local districts with mechanisms for 
furthering their own reform work. 

• Sustaining the progress that has been made 

First and foremost, the most successful structures of the project - the Teacher Partners 
and Resource Collaboratives - need to continue their work beyond the limited time of 
the grant. Several of the Resource Collaboratives have already found secure homes 
within the departments of education at the universities in which they are housed, and 
several are becoming more integrated into the programs offered within those 
departments. But this is not yet true across the board. The Resource Collaboratives 
were not initially part and parcel of the universities in which they were housed, nor 
were the Resource Collaborative staff people hired from the university. Thus, 
additional attention needs to be paid to the relationship between the Resource 
Collaboratives and their host universities. 
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Related to that, ARSI could perhaps find additional ways to work more closely with 
both the universities which house the Resource Collaboratives and local community 
colleges throughout the region. ARSI could do more to increase the linkages to higher 
education and other initiatives within the various states to use as resources for technical 
assistance, and professional development. ARSI started out as broker, then moved to 
being a provider of services. Now that ARSI has built the infrastructure and helped 
districts be more savvy consumers, the role of broker may be more appropriate. 

Second, some districts find it difficult to sustain the position of the Teacher Partners 
without additional outside funding. Moreover, some Teacher Partners struggle to teach 
half time and do ARSI half time - they find themselves working two full time jobs 
instead of one. As one Teacher Partner stated: 

I teach math half a day and am a Teacher Partner the other half. I am suppceed to have 
two half-time jobs, that is what I am supposed to do. The reality is, I have two full-time 
jobs and a half a day to do each one of those. 

Teacher Partners need more release time to fully immerse themselves in their role as 
lead teachers, and to become part of the funded structure of the districts in which they 
work. Tire best cases we saw are those in which Teacher Partners worked themselves 
into a district-level job as a full-time resource teacher and district specialist. 

• Finding ways to develop community understanding and support 

Community engagement remains one of the greatest challenges to the project, even 
though everyone agrees on the importance of involving the community in making real 
changes come about in Appalachia. There are multiple dimensions to be found within 
the whole idea of "Community engagement". One aspect certainly involves finding 
ways to help key community leaders understand the specific work of ARSI - and then 
support it. Another aspect involves finding ways to use the ARSI work to generate 
greater commitment of the community to education more broadly - helping them see 
the real community benefits that might accrue from improved teaching and learning in 
their schools. Finally, there is the fact that ARSI itself can learn a great deal from a well- 
designed process of interacting with local community members. 

ARSI's work to date has certainly highlighted the complexity of community 
engagement. ARSI learned during the project that it can not use just one model or one 
approach to community engagement. Sometimes it works best when ARSI creates its 
own community engagement team; there are many other times when it is better for 
ARSI to infuse its work into existing school-community committees. In this area we 
think best practices and case studies of successes still need to be gathered and shared, 
and a more sophisticated, but also concrete, vision about what "community 
engagement" really means still needs to be developed. 
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Working within existing state contexts 



Also, ARSI works in a problematic environment of trying to institute reforms within six 
very strong and different state contexts. These state contexts of accountability and 
testing are intense forces that shape the entire educational endeavor. While project staff 
indicate that they have had good support from the six states, it is still challenging to be 
a strong outside voice for reform efforts within these already existing noisy 
environments. 

Related to that, we found that most districts participating in ARSI still need concrete 
visions for math and science reforms that go beyond raising test scores. While ARSI has 
tried to turn districts' attention to the national standards, the state testing programs and 
accountability issues are so pervasive that it is difficult for districts to develop strong 
shared visions of improvement efforts that look beyond immediate test results. 

In addition, most of the states with which ARSI works have spent the last five years 
investing substantial sums of money in upgrading the technology in all of their districts. 
However, teachers and administrators in the districts we visited still need a great deal 
of help in learning how best to use this technology in the service of improving math and 
science instruction. Technology tends to be used as an end in itself, or as a tool for 
reinforcing basic skills, rather than as an integrated part of standards-based instruction 
in mathematics and science. 

• Strengthening curriculum 

In general, Appalachian districts have been weak in terms of curricular support. Often 
teachers have been left without curriculum and without access to high quality 
instructional materials. While ARSI has influenced districts' knowledge of and exposure 
to "exemplary" curriculum, there is little history or knowledge that resides within 
districts as to the whole process of adopting and implementing curricular programs. 
Despite the fact that a wealth of science and mathematics materials funded by NSF and 
others now exist - materials that are supportive of the vision of science and math 
teaching and learning espoused in the national standards - we saw little evidence of 
wide-scale use of these "exemplary" curricula in ARSI districts. We believe that putting 
good curricular programs and materials in place would make a significant difference in 
the districts; conversely, we believe that without such programs and materials the 
progress made in these districts will remain limited to practices of the best teacher 
leaders. 
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Summary 



The ARSI story, we believe, is both inspiring and daunting. It is inspiring in the way 
that it has empowered local teachers, administrators, and communities to work hard to 
improve their own programs. It is daunting in that it shows that there are no shortcuts 
or substitutes for the long-term process of building up the capacity these districts need 
to provide their students with high-quality mathematics, science and technology 
programs. We hope that this telling of the ARSI story will help others learn from their 
efforts, and help all of us understand better the needs that exist and the nature of the 
work that is required to address them. 
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